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We determine the complexity of several constraint satisfaction problems using the heuristic al- 
gorithm, WalkSAT. At large sizes A^, the complexity increases exponentially with N in all cases. 
Perhaps surprisingly, out of all the models studied, the hardest for WalkSAT is the one for which 
there is a polynomial time algorithm. 
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I. INTRODUCTION 

There is considerable interest, in different fields of sci- 
ence, in finding efficient methods to solve optimization 
problems. For a few such problems there are clever al- 
gorithms which enable one to find the solution, for all 
cases, in a time which only grows with a power of the 
size N of the problem. Problems with such a polynomial 
time algorithm are said to be in the complexity class P. 
In many cases of interest, however, no polynomial time 
algorithm is known, though a solution, if given, can be 
verified in polynomial time. Decision problems for which 
which "yes" instances can be verified in polynomial time 
are said to be [l| in complexity class NP. There is no proof 
that P ^ NP, though it is generally assumed that they are 
different, i.e. there exist problems which cannot be solved 
in polynomial time algorithm, at least in the worst case. 
There is a subset of NP problems which have the prop- 
erty that any problem in NP can be mapped into them 
in polynomial time. These are called NP-complete T]. 
Consequently if a polynomial time algorithm could be 
found for one NP-complete problem, all problems in NP 
could be solved in polynomial time and so we would have 
P = NP (which, as stated above, is felt to be unlikely). 

Several heuristic algorithms to solve optimization 
problems have been proposed. ("Heuristic" means that 
the answer provided is not guaranteed to be exact. This 
is in contrast to "complete" algorithms which guaran- 
tee to find the solution, if one exists.) One well known 
example is simulated annealing [H (SA) in which an arti- 
ficial temperature is introduced and gradually set to zero. 
Another popular algorithm is WalkSAT ^] which is sim- 
ilar in spirit to simulated annealing in that both make 
moves which reduce the "energy" , but also sometimes 
make moves which increase it to avoid being trapped in 
the nearest local minimum. 

It has also been proposed to solve optimization prob- 
lems on a quantum computer using the quantum adia- 
batic algorithm (QAA) 4], which is based on quantum 
annealing [s^. To assess whether a quantum computer 
could solve optimization problems more efficiently than 
a classical computer, it is valuable to compare the effi- 
ciency of the QAA to solve a range of optimization prob- 



lems with that of classical heuristic algorithms, in par- 
ticular to see whether those problems which are harder 
classically are also harder quantum mechanically. 

As part of this project we report here results of the 
efficiency of a classical algorithm to study several opti- 
mization problems of the "constraint satisfaction" type. 
We chose WalkSAT rather than SA because the imple- 
mentation is simpler since there is just one parameter 
(the strength of the "noise") whereas in SA one has to 
decide on the whole annealing schedule of temperature 
against time. It is not obvious how to choose the best 
annealing schedule, and different choices could lead to 
significant differences in the number of sweeps needed to 
find the ground state, which would be a disadvantage for 
us. Also, there is a publicly available code for WalkSAT 
which can be downloaded and easily compiled on differ- 
ent machines. 

We study three NP-complete models and two versions 
of one model that is in the P class (known as XORS AT) . 
For WalkSAT, all models we study require a computer 
time which increases exponentially with N. Some are 
harder than others, in that the coefficient of N in the ex- 
ponent is larger. Curiously the hardest of all is XORSAT 
even though there exists a polynomial time algorithm for 
this problem. Hence, the difficulty of a problem using 
a heuristic algorithm does not, in general, depend on 
whether it is in the P or NP complexity class. Interest- 
ingly, it is also known that XORSAT is very hard using 
the QAA 0. 

The plan of this paper is as follows. Section |TT] de- 
scribes the four models that will be studied. Results are 
presented in Sec. Illll and our conclusions summarized in 
SecCVl 



II. MODELS 

We shall consider problems of the "constraint satisfac- 
tion" type, in which there are N bits (or equivalently 
Ising spins) and AI "clauses" where each clause is a logi- 
cal condition on a small number of randomly chosen bits. 
A configuration of the bits is a "satisfying assignment" if 
it satisfies all the clauses. Frequently, in statistical me- 
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N 


16 


32 


64 


128 


192 


256 


M 


12 


23 


44 


86 


126 


166 


a 


0.7500 


0.7188 


0.6875 


0.6719 


0.6563 


0.6484 



TABLE I: The number of clauses M and number of bits A'' 
used in the study of the Exact Cover problem, from Ref. Ql . 
The ratio M/N is expected to approach the value at the sat- 
isfiability phase transition Qs — 0.625 [lol . [Tll | for N oo. 

chanics approaches to these problems, one converts each 
clause to an energy function, which depends on the bits 
in the clause, such that the energy is zero if the clause 
is satisfied and is positive if it is not. However, we will 
not need to do this here since WalkSAT uses the logical 
structure of the clauses rather than an energy function. 

Clearly, it is easy to satisfy all clauses if the ratio 
a = M/N is small enough. In fact one expects an expo- 
nentially large number of satisfying assignments in this 
region. Conversely, if M/N is very large, with high prob- 
ability there will be a conflict between different clauses. 
Hence there is a "satisfiability transition" at some value 
as where the number of satisfying assignments goes to 
zero. It is believed that it is particularly hard to solve 
satisfiability problems close to the transition Jj, and so 
we will work in this region. Furthermore, when study- 
ing the efficiency of the QAA numerically 0, H, Q, it 
is convenient to consider instances with a unique satis- 
fying assignment (USA). Since we intend eventually to 
compare the results presented here with results for the 
QAA, here we will also only consider instances with a 
USA (which of courses, forces the system to be close the 
transition). 

We now discuss the different models that will be inves- 
tigated in this paper. 



A. Exact Cover (Unlocked l-in-3 SAT) 

Several numerical studies of the QAA [1, H, 0] consider 
the "Exact Cover" problem, in which each clause consists 
of three bits chosen randomly, and the clause is satisfied if 
one bit is one and the others are zero. In order to increase 
the probability of a USA, Young et al. [1, Q removed 
isolated bits, and clauses which are only connected to the 
others by one bit. Here we use exactly the same instances 
as in Refs. [1, Q. For each value of N, the number of 
clauses M is chosen to maximize the probability of a 
USA. The values of N and M are shown in Table H It 
is found that the probability of a USA decreases with N, 
apparently exponentially. 

This Exact Cover problem is sometimes called l-in-3 
SAT, for obvious reasons. In subsections IIIBI and III C[ 
we will discuss models with a special property called 
"locked", defined by Zdeborova and Mezard [Ulil. To 
distinguish the present model from the locked models, we 
will refer to it as "unlocked l-in-3 SAT" from now on. 



B. Locked l-in-3 SAT 

Recently Zdeborova and Mezard have proposed 

that it is useful to study a set of models, which they call 
"locked" , which have the following two properties: 

1. Every variable is in at least two clauses. 

2. Whether or not a clause is satisfied only depends on 
the sum of the bits in it (occupation problem) , and 
two successive values of the sum are not allowed. 
Thus l-or-3-in-4 is allowed, but l-or-2-in-4 is not. 

It follows that one can not get from one satisfying as- 
signment to another by flipping a single bit. In fact, 
Zdeborova and Mezard argue that typically order InA^ 
bits needs to be flipped. They also argue that, locked in- 
stances are analytically "simple" (or at least simpler than 
previously studied models such as random K-SAT) but 
are computationally hard. They are therefore eminently 
suitable as benchmarks. 

If the sites are chosen at random to form the clauses, 
the distribution of the degree of the sites (i.e. the number 
of clauses involving a site) would be Poissonian. How- 
ever, locked instances have a minimum degree of two, so 
instead we use a truncated Poissonian distribution [13| 
which is Poissonian except that the probabilities for zero 
and one are set to zero. We fix the ratio M/N to be the 
critical value for the satisfiability transition. According 
to Table I of Ref. [ll], this is equal to = 0.789. Since 
M has to be an integer we take M to be the nearest 
integer to agN. 

Having generated these instances we run them 
through a (compl ete) Davis-Putnam-Logemann- 
Loveland (DPLL) [ij, [l^ code to select those 
with a unique satisfying assignment (USA). 

The probability of a USA only decreases slowly with N 
and may tend to a non-zero value as N ^ oo, see Fig. [T] 
This is in contrast to the unlocked instances in Sec. Ill Al 
for which the probability decreases exponentially with 
N. The locked problem therefore has the advantage that 
instances with a USA should be a good representation of 
randomly chosen instances. 



C. Locked 2-in-4 SAT 

We also consider locked 2-in-4 instances, in which a 
clause has four bits, and a clause is satisfied if two are 
zero and two are one. Unlike the other models discussed 
in this paper, this one has a symmetry under flipping all 
the bits. 

We fix the ratio M/N to be the critical value for the 
satisfiability transition. According to Table I of Ref. [l^ , 
this is equal to as = 0.707. Since M has to be an integer 
we take M to be the nearest integer to UgN . As with the 
locked l-in-3 instances in Sec. IIIBI the probability of a 
USA seems to tend to a non-zero value for iV — s- oo, see 
Fig.H 
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FIG. 1: (Color online) Probability of a unique satisfying as- 
signment (USA) as a function of for locked instances of 
l-in-3 SAT at the satisfiability threshold. The line is a guide 
to the eye. 



FIG. 2: (Color online) Probability of a unique satisfying as- 
signment (USA) as a function of 1/7V for locked instances of 
2-in-4 SAT at the satisfiability threshold. The line is a guide 
to the eye. 



D. XORSAT 

The exclusive-or of a set of bits is their sum (mod 2). 
In the K-XORSAT problem, K bits are chosen to form 
a clause and the clause is satisfied if their sum (mod 2) 
is a specified value (either or 1). Again, the problem 
to be solved is whether there is an assignment of the 
N bits which satisfies all M clauses. In fact, since the 
problem just involves linear algebra (mod 2) the satisfi- 
ability problem can be solved in polynomial time using, 
for example, Gaussian elimination. However, if there is 
not a satisfying assignment, there is no known polyno- 
mial time algorithm to determine the minimal number of 
unsatisfied clauses, a problem known as MAX- XORSAT. 

For XORSAT instances with a USA, it is not difficult 
to show that one can gauge transform any instance into 
one in which the sum of the bits of every clause is equal to 
(mod 2). The USA is then all bits equal to 0, (a "ferro- 
magnetic" ground state in statistical physics language). 
Although this ground state is "trivial" , we shall see that 
it is very hard to find using a heuristic algorithm. 

Here we will take the case oi K ~ 3, and study two 
variants of the model. 



1. 3-regular 3-XORSAT 

Firstly, we will follow Jorg et al. Q in taking the "3- 
regular" case where every bit is in exactly three clauses, 
a model which turns out to be precisely at the satis- 
fiability threshold. We denote this model as 3-regular 



3-XORSAT. As usual, we consider instances with a 
USA. Fortunately, these are a non-zero fraction, about 
0.285 6J, of the total, so the USA instances should be a 
good representation of randomly chosen ones. Note that 
M = N for this model. 



2. 3-XORSAT 

Secondly, following the suggestion of Lenka Zdeborova, 
we will consider instances in which the distribution of the 
number of clauses attached to a bit is truncated Poisso- 
nian with mean degree three. For this model, which we 
denote as 3-XORSAT, we find numerically that the frac- 
tion of instances with a USA is also non-zero, about 0.25. 
As usual, we restrict our attention to these instances. 



III. RESULTS 

We study the models in Sec. E] using the WalkSAT 
algorithm. WalkSAT picks at random a clause which is 
currently unsatisfied and flips a variable in that clause. 
With some probability the variable is chosen to be the 
one which causes the fewest previously satisfied clauses to 
become unsatisfied, and otherwise it is chosen at random. 
The probability that the variable is chosen at random is 
called the "noise parameter". We know that there is a 
(unique) satisfying assignment for each instance (since we 
selected instances to have this condition using a DPLL 
algorithm). We determine how many elementary Walk- 
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model 




unlocked l-in-3 SAT 


0.027 


locked l-in-3 SAT 


0.052 


locked 2-in-4 SAT 


0.086 


3-XORSAT 


0.116 


3-regular 3-XORSAT 


0.124 



TABLE II: The rate of exponential increase of the number of 
flips, according to Eq. © for the different models using the 
default value of the noise. 

SAT "flips" are needed to find it, for the different models 
for a range of sizes. 

The overall logical condition which must be satisfied 
is the logical AND of each clause. Most algorithms for 
solving satisfiability problems, including WalkSAT, re- 
quire that the problem is expressed in conjunctive nor- 
mal form (cnf), in which each clause is written entirely 
in terms of logical OR's. Note that OR just excludes 
one possible state of the bits. The cnf representation of 
the problems studied here is not unique. For example, in 
l-in-3 SAT, a clause requires xi + X2 + = 1, i.e. one 
of the bits is 1 and the others are (so there are 3 al- 
lowed configurations) . This has to be written as a logical 
AND of several cnf clauses (each of which is comprised 
of OR's). A natural choice is to use five clauses as 



(xi V a;2 V X3) A (-1X1 V -1X2 V -'a;3)A 
(xi V -1X2 V -1X3) A (-1X1 V X2 V -'X3)A 
(-■xi V -1X2 V X3) , 



(1) 
(2) 
(3) 



in which each clause disallows one of the 8 — 3 {— 5) 
forbidden configurations. However, we can actually com- 
bine two of the clauses together using a clause with only 
two variables, for example as 

(xiVx2Vx3)A(-xi V-X2)A (4) 
(xi V -1X2 V -1x3) A (-ixi V X2 V -1x3) . (5) 

We always chose the cnf representation that used the 
smallest number of clauses. 

First, we show results using the default value of the 
noise parameter (0.5). Figure [3] plots the median number 
of "flips" to find a solution as a function of N for the 
different models. Note the logarithmic vertical scale. The 
data fits a straight line at large N indicating a number of 
flips (which is proportional to the CPU time) increasing 
exponentially, as expected. 

We see that the easiest model is unlocked l-in-3 SAT 
(Exact Cover), while the hardest is 3-XORSAT. Both 
versions of 3-XORSAT are harder than any of the other 
problems that we looked at, with the 3-regular version 
being somewhat harder than the version with a Poisso- 
nian degree distribution. 

Writing the median number of flips as 




3-reg 3-XORSAT i-e- 
3-XORSAT i-B- 
locked 2-in-4 SAT 1— v 
locked 1-in-3 SAT 1— ■- 
unlocked 1-in-3 SAT i-^A- 
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FIG. 3: (Color online) A log-lin plot of the median number 
of flips needed to solve the various problems as a function 
of using WalkSAT with the default noise parameter. The 
straight lines are fits to Eq. Q for the larger sizes. 



WalkSAT (opt) 




3-reg 3-XORSAT 

3-XORSAT " 
locked 2-in-4 SAT 1— ^ 
locked 1-in-3 SAT 1— ■- 
unlocked 1-in-3 SAT i-^^ 



10 



50 



100 



150 
N 



200 



250 



300 



iVflips = Ae'^^ 



(6) 



FIG. 4: (Color online) A log-lin plot of the median number 
of flips needed to solve the various problems as a function 
of A'^ using WalkSAT with optimized noise parameters. The 
straight lines are fits to Eq. ^ for the larger sizes. The results 
are quite similar to those for the default noise shown in Fig. [3] 



we present the values of /i in Table HIl It is curious that 
the one problem in the set (XORSAT) which is in the 
polynomial time complexity class, P, is the hardest for a 
heuristic algorithm. 

We have also investigated the extent to which Walk- 
SAT can be improved by optimizing with respect to the 
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model 




unlocked l-in-3 SAT 


0.021 


locked l-in-3 SAT 


0.034 


locked 2-in-4 SAT 


0.053 


3-XORSAT 


0.085 


3-regular 3-XORSAT 


0.107 



TABLE III: The rate of exponential increase of the number 
of flips, according to Eq. © for the different models using op- 
timized values of the noise. These values for jj, are somewhat 
smaller than those for unoptimized noise, in Table [III but the 
overall trend between the different models is the same. 

noise parameter. For each size, and each instance, we 
try different values of the noise and home in on the value 
which minimizes the number of flips. Results for the 
median number of flips are shown in Fig. 21 Naturally, 
the numbers are a bit smaller than for the unoptimized 
case in Fig. |3l but we still see an exponential growth at 
large sizes, and the ordering of the difficulty of the differ- 
ent models is the same (e.g. unlocked l-in-3 SAT is the 
easiest and and the two versions of 3-XORSAT are the 
hardest). The fits in Fig. SI correspond to the coefficients 
of N in the exponential growth shown in Table IIIIl see 
Eq. ([5]). Unsurprisingly, these are somewhat less than 
the values for the unoptimized case shown in Table [III 



IV. CONCLUSIONS 

We have studied the complexity of the WalkSAT al- 
gorithm for four constraint satisfaction problems, three 



of them in the NP complexity class and one in the P 
complexity class. All show exponential complexity for 
large sizes. Curiously, the hardest problem for WalkSAT 
is the one in P. It will be interesting, in future work, to 
compare the relative hardness of these problems for the 
classical WalkSAT algorithm that we found here, with 
their relative hardness when using the quantum adiabatic 
algorithm. 
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